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T 1 

We construct a class ol two Higgs doublets models with a 4th sequential generation of fermions 
that may effectively accommodate the low energy characteristics and phenomenology of a dynamical 
electroweak symmetry breaking scenario which is triggered by the condensates of the 4th family 
fermions. In particular, we single out the heavy quarks by coupling the "heavier" Higgs doublet 
(<3?h) which possesses a much larger VEV only to them while the "lighter" doublet ($^) couples only 
to the light fermions. We study the constraints on these models from precision electroweak data as 
well as from flavor data. We also discuss some distinct new features that have direct consequences 
£f~^ ■ on the production and decays of the 4th family quarks and leptons in high energy colliders; in 

particular the conventional search strategies for t' and b' may need to be significantly revised. 
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I. INTRODUCTION 

One of the most studied, yet unresolved theoretical puzzles in modern particle physics is the origin of Electro Weak 
, symmetry breaking (EWSB). Indeed, it is widely anticipated that the LHC will provide us with crucial answers 
^\ • regarding the underlying nature of EWSB: is the Higgs a fundamental scalar needing protection from SUSY or is it a 
CO ! composite object. In the Standard Model (SM), EWSB is triggered by the Higgs mechanism, which assumes a single 
^sO ■ fundamental scalar, the Higgs, with a mass at the EW-scale. This leads to the long standing difficulty known as the 
j hierarchy problem: the presence of a fundamental EW-scale seems unnatural since there is a problem of stabilizing 
the Higgs mass against radiative corrections without introducing a cutoff to the theory at the nearby TeV scale. The 
hierarchy problem, which is usually being interpreted as evidence for new TeV-scale physics, has fueled much scientific 
, effort in the past several decades, both in theory and in experiment. 

■ Furthermore, recent flavor physics studies have revealed some degree of tension in the CKM fits for the SM with 
3 generations [iHjl. For example, there are indications that the "predicted" value of sin 2/3 is larger than the value 
measured directly via the "gold-plated" ipK s mode by as much as ~ 3.3a- [6]. On the other hand, the announced 
CDF and DO results on the CP asymmetry in B s — > ijjcj) (at a higher luminosity around 6/fb) are larger than the 
SM prediction by about la [7J, and at the same time, they find a surprisingly large CP-asymmetry in the same-sign 
dimuons signal, which they attribute primarily to a s sl - the semileptonic asymmetry in B s — > X s fiv Q. 

Interestingly, perhaps the simplest variant of the SM, known as the SM4, in which only a 4th sequential generation 
of fermion doublets is added to the the ory (fo r reviews see can address some of the theoretical challenges 

associated with the hierarchy problem |13l4l5j and can readily account for the CKM anomalies mentioned above 
[l6l - r24| . In particular, as was suggested over two decades ago, a heavy 4th generation fermion may trigger dynamical 
EWSB The picture that arises in this scenario is of new heavy fermions which have large Yukawa couplings 

that are driven to a Landau pole or a fixed point (which acts as a cutoff), possibly at the nearby TeV scale [14l Il5|. 
Consequently, some form of strong dynamics and/or compositeness may occur and the Higgs particles are viewed as 
composites primarily of the 4th generation fermions (see e.g., [25U27j ] ) . with condensates < Q'it' R >=^ 0, < Q'ib' R 
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(and possibly also < L' L v' R 0, < L' l t' r >^ 0), which induce EWSB and generate a dynamical mass for the 
condensing fermions. As for the CKM anomalies, the two extra phases that the SM4 possesses can give rise to a host 
of non-standard CP asymmetries |2S . |29 and, in addition, can significantly ameliorate the difficulties with regard to 
baryogenesis that the SM has [HlH E|- 

Indeed, recent searches for 4th generation heavy quarks by the CDF Collaboration have found that m t ' , my £ 350 
GeV 32] - in support of the compositeness scenario. Thus, any theory that contains these heavy fermionic states 
is inevitably cutoff at the near by TeV-scale where compositeness is expected to occur. As was realized already 20 
years ago by Luty 33], the compositeness picture which emerges in this case may be more naturally embedded at 
low energies in multi-Higgs theories, since one should expect several composite scalars to emerge as manifestations 
of the different possible bound states of the fundamental heavy fermions. This idea was further studied recently in 
[27l I3414371 ] . Moreover, as will be shown in this paper, the addition of more scalar doublets relaxes the constraints 
from precision EW data (PEWD) (see also [36]), and allows for interesting new dynamics associated with the 4th 
generation fermions that can be tested at high energy colliders. 

Adopting this viewpoint, in section II we construct a class of models with two scalar doublets and four generations 
of fermions that can serve as effective low energy frameworks and capture the key ingredients of the TeV-scale 
compositeness picture, by giving a special status to the heavy fermionic states. We then analyze in section III the 
constraints on these models from PEWD and from flavor physics in b-quark systems, and in section IV we discuss 
some of the new distinct phenomenological consequences of our multi-Higgs setup for collider searches of the 4th 
generation fermions. Finally, in section V we summarize our findings. 



II. TWO HIGGS DOUBLET MODELS FOR THE 4TH GENERATION FERMIONS - 4G2HDMS 

Recall that in a type II 2HDM (see (38[) one Higgs doublet couples only to the the up-quarks while the 2nd Higgs 
doublet couples to the down-quarks. It is straight forward to extend such a setup to the case of a 4th generation 
fermion doublet - this was considered in [36l [39ll40| and within a SUSY framework in [30l l4l| - |43l | . 

Our aim here is to construct a new class of two Higgs doublet models (2HDMs) that can serve as a viable low energy 
effective framework for models of 4th generation condensation. Thus, in analogy with the 2HDM setup proposed in 
[H|], we construct our 4G2HDMs using different Yukawa textures than the "standard" 2HDM of type II. In particular, 
in our 4G2HDMs one of the Higgs fields (call it the "heavier" field) couples only to heavy fermionic states, while the 
second Higgs field (the "lighter" field) is responsible for the mass generation of all other (lighter) fermions. In this 
way, the heavier field may be viewed as a q'q' composite with a condensate < q' q' >=/= 0. 

The Higgs potential is a general 2HDM one [38[ and the Yukawa interaction Lagrangian of the quark sector is 
defined as: 



Cy = -Ql (*iF ■ (/ - I^ d ) + $ h F ■ T^ d ) d R - Ql • (/ - 2^) + $ h G ■ Z^" 9 ") u R + h.c. , (1) 

where Jl( R ) are left (right) -handed fermion fields, Ql is the left-handed SU(2) quark doublet and F,G are general 
4x4 Yukawa matrices in flavor space. Also, $^ are the Higgs doublets: 




/ is the identity matrix and Xq q q (q = d, u) are diagonal 4x4 matrices defined by Iq q = diag (0, 0, a q , /3 q ). 

The Yukawa texture of fTJ) can be realized in terms of a ^-symmetry under which the fields transform as follows: 

$ e -> -$ e , $ ft -> Q L -> +Q L , 

d R -t -d R (d = d, s), u R -t -u R (u =u,c), 

b R -+ {~l) 1+ad b R , b' R (-l) 1+ ^b' R , 

t R ^(-l) 1+a «t R , t' R ^(-iy + ^t' R . (2) 

One can thus construct several models in which the Yukawa interactions of the heavy fermionic states have a non- 
trivial structure, possibly associated with the compositeness scenario. Three particularly interesting models which we 
will study in this paper are: 

• 4G2HDM-I: (aa, /3d,a u , f3 u ) = (0,1,0,1). In this case &h gives masses only to t' and b' , while generates 
masses for all other quarks (including the top-quark). 
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• 4G2HDM-II: (a d , fid, a u , fiu) = (1,1,1,1)- In this case the heavy condensate is responsible for the mass 
generation of the heavy quarks states of both the 3rd and 4th generation quarks, whereas $g generates masses 
for the light quarks of the 1st and 2nd generations. 

• 4G2HDM-III: (a d , (3 d , «u, Pu) = (0, 1, 1, 1). In this case nit, my, m t ' oc t>h, so that only quarks with masses at 
the EW-scale are coupled to the heavy doublet 

The above 3 models represent, in our view, the minimal set of multi-Higgs frameworks that capture the compos- 
iteness scenarios associated with the heavy 4th generation fermions. Defining tan/3 = Vhjvt, in the 4G2HDM-I we 
expect tan ,3 ~ m q ,/m t ~ 0(1) (q' = t' or b'), while for the 4G2HDM-II and 4G2HDM-III models, tan/3 > 1 seems 
to be a more natural choice. 

As mentioned earlier, the construction of our 4G2HDM models was inspired in part by the 2HDM "for the top- 
quark" , which was introduced by Das and Kao in (44| and which was designed to give an effective explanation for 
the large top-quark mass via Vh S> V£. However, there is a fundamental difference between our 4G2HDMs and 
the Das and Kao 2HDM: the Das and Kao model which was constructed with three fermion generations has no 
new heavy fermions (the heavier Higgs doublet, couples only to the top-quark). Thus, without the new heavy 
fermionic degrees of freedom, the top-quark Yukawa coupling remains perturbative up to the Planck scale, so that 
their 2HDM does not have a natural low-energy cutoff as one would expect for the condensation picture. On the other 
hand, in our 4G2HDMs the strong Yukawa couplings of the heavier Higgs field to the new heavy 4th family fermions 
reaches a Landau pole at the near by TeV-scale, thus signaling new physics - possibly in the form of compositeness. 
Alternatively, our framework might be more naturally embedded into weakly coupled theories in 5 dimensions, see 
e.g., ii. 

From the point of view of the leptonic sector, the type-I 4G2HDM is the more natural underlying setup that can 
effectively accommodate the heavy masses of the 4th generation neutrino v' . In particular, recall that the current 
bounds on m v i [4!| indicate that v' should have a mass at least at the EW-scale. The main glaring problem for the 
SM4 is the fact that it does not address the origin of such a heavy mass for v' [48]. On the other hand, within our 
4G2HDM-I the heaviness of the 4th generation leptons (with respect to the lighter three generations) is effectively 
accommodated by coupling them to the heavy Higgs doublet. This setup for the leptonic sector might also be an 



effective underlying description of more elaborate construction in models of warped extra dimensions, see e.g., 45 1. 

The physical Higgs fields H and h, H, A (h and H are the lighter and heavier CP-even neutral states, respectively, 
and A is the neutral CP-odd state) are obtained by diagonalizing the neutral and charged Higgs mass matrices: 

*J = c G+ - s H+ , 
$- = spG + +c p H+ , 
$° = c a H - s a h + i(c{iG - spA) , 
= s a H + c a h + i (s G° + cpA) , 

(3) 

where G + ,G° are the goldstone bosons, cp, sp = cos/3, sin/3, c Q ,s Q = cos a, sina and a is the mixing angle in the 
CP-even neutral Higgs sector. 

The Yukawa interactions between the physical Higgs bosons and quark states are then given by: 

£(h mj ) = ^q^m^Sij - (^ + ^y[m^R + m^f i L]jq j h , (4) 

C(Hq iqj ) = -^—q % |-m 9i ^.% + ^ - • [m^lR + m^L] J Qj H , (5) 

jC(Aqiqj) = -ilq^—qt {m qz tanfi^Stj - (tan/3 + cot/3) • [m^Ey-R - m qj T,j*L] } q 3 A , (6) 
CiH+u.dj) = -jd — iii { [m d tan ,3 ■ V Uid] - m dk (tan/3 + cot/3) ■ VifcEjL] R 

+ [-m Ui tan ■ V Uidj + m Uk (tan /3 + cot 0) ■ E^V kj ] L) d 3 H+ , (7) 

where q = d or u for down or up-quarks with weak Isospin I d = — | and I u = +i, respectively, and R(L) = 
5 (1 + (— )75). Also, V is the 4x4 CKM matrix and E d (E u ) are new mixing matrices in the down(up)-quark sectors, 
obtained after diagonalizing the quarks mass matrices: 
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^ij(ttd,0d,D R ) = a d D* R , 3i £>H,3j ■ + PdD* R M D RAj > 

^ll( a u,(3u, Ur) = Ct u U R 3l UR y3 j + (3 u Ur 4l URAj , 



(8) 



where Dr, Ur are the rotation (unitary) matrices of the right-handed down and up-quarks, respectively. Notice that 
E" and E d depend only on the elements of the 3rd and 4th rows of Ur and Dr, respectively, and on whether a q 
and/or j3 q are "turned on". For example, in model 4G2HDM-I, for which (ay, j3 d , a u , /?«) = (0,1,0,1), only the 4th 
row elements of Ur and Dr are relevant. 

Recall that in standard frameworks such as the single-Higgs SM4 or 2HDMs of types I and II (38|, the right- 
handed mixing matrices Ur and Dr are non-physical in the sense that they are "rotated away" in the diagonalization 
procedure of the quark masses. On the other hand, in our 4G2HDMs some elements of these matrices can, in principle, 
be measured in Higgs-fermion systems, as we will later show. One can, thus, treat these matrices as unknowns, by 
expressing physical observables in terms of the elements of the 3rd and 4th rows of Ur and Dr, or study there 
properties under some theoretically motivated parameterization. In particular, inspired by the working assumption 
of our 4G2HDMs and by the observed flavor pattern in the up and down-quark sectors, we may assume the following 
structure (see also [44| for the 3x3 case): 



D R = 



Ur 



( cos ds -sin9 ds 

sin 9 ds cos ds 
e s 
\ 

/ cos6> nc -sin6» nc 
sin UC cos UC 
e c 
\ 



sinews cos u bb ,e~ 
sin ds sin6>M/e*e" 
cos Qw 
smO w e l5b 

sin UC cos tt 'e* 
■ sin One smO t t'e* e ~ 
cos Otv 



i6t 



- COS ds COS 0wt* s \ 

cos ds sin w e k s e' lSh 
-smO w e~ iSb 
cos W 



-cos 6 UC cos tt 'e* 9 ^ 
cos UC sm0tt'£* c e~ lSt 



(9) 



sin w e 



i5 t 



■smv tt >e 
cos On' 



(10) 



/ 



— e lSa , so that unitarity of Dr and Ur is restored at 1st order in e s and e c , respectively. 

<< 1,W Ur and Dr simplify to (similar textures can be 



mb u m t 

In the limit sinO uc ~ m u /m c << 1 and svnOd, 
found in Randall-Sundrum warped models of flavor [46|, |47|): 



D R = 



where we have further defined 
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e b = smO bb ,e 
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(11) 



(12) 

We thus obtain for the E mixing matrices in Eq.[8](in each element keeping only the leading terms in e q , q — s,c, b, t): 

\ 











Ctd\f-s 











-aye s e 6 



o^(i-¥) 

(Ai - a d )e b (1 - ^f) a d |e b | 2 + /3 d (l-^ 



0a d e s (l-^f) a d (l-kf) + f3 d \e b \ 2 {p d - a d )e% (l - ^ 



(13) 



/ 



and similarly for E" by replacing a d ,p d — > a u , f3 u and e s , £6 — > e c ,et- 

A natural choice which we will adopt in some instances below is: \e t \ = sm0 t t> ~ m t /m t i and |e&| = sin#t,b< 
mb/mb'. 



[1] The mixing angles $ uc and dds have no effect in our models as they enter only in the 1st and 2nd rows of Ur and Dr which have no 
physical outcome. 
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III. CONSTRAINTS ON THE 4G2HDMS 

We now consider constraints from PEWD and from flavor physics in b-quark systems; namely B — > X s -f and B q — B q 
(q = d, s) mixing. The PEWD constraints can be divided into the effects of the heavy new physics which does and 
does not couple directly to the SM ordinary fermions. For the former we consider constraints from Z — > bb, which is 
mainly sensitive to the H + t'b and W + t'b couplings in our models. The effects which do not involve direct couplings 
to the ordinary fermions, are analyzed by the quantum oblique corrections to the gauge-bosons 2-point functions, 
which can be parameterized in terms of the oblique parameters S,T and U [501 ] - It should be noted that, as far as the 
oblique parameters are concerned, the contribution from our 4G2HDMs is identical at the 1-loop level to that of any 
2HDM, since the new Hff Yukawa interactions in our models do not contribute at 1-loop to the gauge-bosons self 
energies. 

A. B — > Xs'y and B q -B q mixing 

1. B^X sl 

The inclusive radiative decays of the B meson are known to be a very sensitive probe of new physics. Strong 
constraints on new physics from B — > X s j [511453T ] crucially depend on theoretical uncertainties in the SM prediction 
for this decay. At the parton level, the decay process B — > X s j is induced by the flavor changing (FC) decay of the 
b-quark into a strange quark. 

The current experimental world average is given by Q , 

BR[B -> X s j] = (3.55 ± 0.24 ± 0.09) x 10~ 4 . (14) 

In the SM, the calculation of the decay rate is most conveniently performed after decoupling the electroweak bosons 
and the top quark. In the resulting effective theory, the relevant FC weak interactions are given by a linear combination 
of dimension-five and -six operators [54j 

03,4,5,6 = Or^Eg^r^), ^gTnoperators) 

7 = ^lLs L a^b R F^, (Photonic dipole 

167T operator) 

0g = EH^s L a^T a b R G a uu . (g!™ dipole (15) 

167T operator) v ' 

The Wilson coefficients, d, of these operators are perturbatively calculable at the renormalization scale /io ~ 
(m^,mt) and the Renormalization Group Equations (RGE) can be used to evaluate Ci at the scale fib ~ Wb/2. 
Finally, the operator on-shell matrix elements are calculated at Hb- At present, all the relevant Wilson coefficients 
Ci(nb) are known at the Next-to-Next-to-Leading-Order (NNLO) [55-62]. However, the matrix elements of the opera- 
tors Oi consists of perturbative and non-perturbative corrections. As far as the perturbative corrections are concerned, 
they are reduced dramatically after the completion of Next-to-Leading-Order (NLO) and NNLO QCD calculations. 
A further improvement comes from electroweak corrections [631-66]. On the other hand, no satisfactory quantitative 
estimates of all the non-perturbative effects arc available, but they are believed to be « 5% [671 ] . 

In the SM within the leading log approximation, the B — > X s j amplitude is proportional to the (effective) Wilson 
coefficient of the operator O7. The well-known [68| expression for this coefficient reads 

cr s M = ^cf Vo) + !(„»-„*) cf M + E M% (i6) 

i— 1 

where rj = a s ((i ) / a s (p b ) and 

fri = (§§§§ -fffi -f -J4 -0-6494 -0.0380 -0.0185 -0.0057) . (17) 
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Separating the charm and top contributions, and neglecting the CKM-suppressed it-quark contribution, eq. (jT6]) 
can be written as 69] 

c! 7 0)cS (f, b )=X c + X u (18) 
where the charm contribution, given by X c , is obtained from eq. (|16[) by the replacement: Cy (fJ>o) ~^ ~ §§ an d 

23 16 8 / 14 16 \ X— \ _ 

which is equivalent to including only charm contributions to the matching conditions for the corresponding operators. 
Analogously, only the top-loop contributes to X t and the expression is given by 



X t = 



1 it I \ ifi 4 _f , s / 14 16 \ 

-^\(xt)v 23 - 3^0 Ot) (/7 23 -V 23 ) , 

(20) 



with xt = (mt(no)/mw) 2 and 



/It /„A _ -3x 3 +2x 2 1t1 i ~22x 3 + 153x 2 -159x+46 



2(a;-l) 4 111 A 1 36(x-l) 3 

?tf T \ _ 3x 2 t . -5z 3 +9:r 2 -30: 
oW — 2(:c-l) 4 111 X + 12(x-l) 3 



Including the perturbative, electroweak and the available non-perturbative corrections, the branching ratio of 
B — > X s j, with an energy cut-off Eq in the i?-meson rest frame, can be written as follows [69(: 



BR[B -► X s7 ]^ b * r ^ ted *■ 



BR[B -> X c eu\ 



v t * s v tb 



Yd 



6a c 



nC 



[P(Eo) + N(E )} , 



(21) 



where a em 
ratio 



.on shell 



[64j . N(Eq) denotes the non-perturbative correction and P(Eq) is given by the perturbative 



T[b -> X sl ] E ^ >Eo 
\V cb /V ub \ 2 T[b -> A u e^] 



6a 



^(^o) • 



(22) 



In their approach (see |69| ) the charmless semileptonic rate has been chosen as the normalization factor in eq. 
whereas C in eq. (1211) is given by 



C 



14, 



cb 



T[B — ► A c eP] 



Furthermore, the perturbative quantity P(Eq) can be written as [6£ 



+ 1 + 



i4 



ln^ r( M o)^+e e 



B(-Bo), 



(23) 



(24) 



where _ftTt contains the top-quark contribution to the b —¥ sj amplitude and K c contains the remaining contributions, 
among which the charm loops are by far the dominant one. Also, the electroweak correction to the b — > sj amplitude is 
denoted in Eq.[53]by e cw and B(E ) is the bremsstrahlung function which contains the effects of b — > sjg and b — > sjqq 
(q = u, d, s) transitions and which is the only E^-dependent part in P(Eq). 
The NLO expression for K t is given by [69j 
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FIG. 1: Examples of one-loop 1PI diagrams that contribute to b — > s"/ in the 4G2HDM, with W-bosons, charged Higgs and 4th 
generation quarks exchanges (ui = u,c,t,t'). 
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(25) 



where the functions A\ (x) and F[ (x) and the expression for K c are given in Ref. [69] . 

For the electroweak (e ew ) and non-perturbative (N(Eq)) corrections in eq. (|21l) we consider the following values 



0.0028 = 0.0075 



e ow « 0.0035 + 0.0012 
N{E ) = 0.0036 ±0.0006. 

Other required inputs which we take from [69| are, 

r(po = mt) = 0.578 ± 0.002 Mi; ± (parametric errors) 

C = 0.575 (1± 0.01 ±0.02 ±0.02) 

a(z) = (0.97 ±0.25) + £(1.01 ± 0.15) 

b{z) = (-0.04 ±0.01) + £(0.09 ± 0.02) , 

where a(z) and b(z) are the z-dependent terms in K c {z = (m c /m b ) 2 , see Eq. 3.7 in 69]). 
With these inputs the NLO prediction for the branching fraction of B — > X s j is [63] 

BR[B -> X s7 ]^>i. 6 GcV = (3.60 ± 0.30) x 10~ 4 . 



(26) 

(27) 
(28) 
(29) 
(30) 



(31) 



In the SM4 there are no new operators other than the ones present in the SM. However, there are extra contributions 
to the Wilson coefficients corresponding to the operators O7 and 0$ from t'-loop |16l - [l3 ]. In our 4G2HDMs the new 
ingredient with respect to the SM4 is the presence of the charged Higgs which gives new contributions to the Wilson 
coefficients of the effective theory. Examples of the 1-loop diagrams that contribute to b — > s-f in our 4G2HDMs are 
given in Fig. [T]' 2 l 

In order to include the charged-Higgs effect we need to compute the new Wilson coefficients at the matching scale /j,q 
(The new H + mdj Yukawa interactions in our models are given in Eq.[7|). At the LO, the charged-Higgs contributions, 
with the i-quark in the loops are given by (see also f70l|). 



SCl o)eff (ti0) 







.6 



(32) 



[2] We are considering only the charged Higgs contributions to b — > 57 and neglecting the flavor changing neutral Higgs 1-loop exchanges, 
which are much smaller in our models due to the very small b — s and V — s transitions as embedded in T, d (see Eq. 1131 1. 
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5C^ eH m = ^4%t) + A Dt F$(y t ), (33) 
and that of t' in the loops are given by 

«7$' e// (/*0) = ^F^(y t ,) + A Dt ,F$(y t ,) , (34) 
where = m ^ M °) ; and the functions Fyg^j/j) are given by [5lT [53l [70j 

s yi(7-5 yi -8y?) y I 2 (3j/,-2) i 

(2A) = 24(^-1)3 + ia^TF lnw ' 

(1) _ ^(2 + 5^-yf) 3^ 2 
8 " 8( W -1)3 4(^-1)*^' 

,-,(2), , yi{3-5yi) y i {3y l -2) 

{yi) = W^? + ^W lnm ' 

Fs (w) " 4(^31)2- 2(^1)3 lny - (35) 

Dropping terms proportional to m s (the strange-quark mass) and also neglecting the terms proportional to Ef,f, oc 
|ef,| 2 (which is expected to be small compared to the leading terms), the factors Au , and Ad , in Eqs.l3"3"landl3"4l 
are given by 



A Ut = ( A Ul -A U2 E t 2 + A /^(^ + ^)S^(A 2 2 S tt -A Ul A u J + ^^<E 2 , 4 , 



Ao t — —A dl A ul + A dl A U2 Yj tt H — -^-(Ad 2 A Ul ~ Ad 2 A U2 'Eu)Y'b'b 

nib v t b 



^ \t A u 2 A d 2 ^t't^b'b + \ -jy* 

Vt rn h \\ s V Vt v t% 



xt '-A U2 Ad 2 ^t't^b'b + \j ^—-^-A^Au^ft, 

A Ut , = (A ul -A U2 S t , t 2 + A /^(^ + ^)S tt ,(<S t , t ,-A 1 ^J + ^^<I] 2 t ,, 

w yv V*, s V t >b y t > X\ b - 



Ad., — —A^Au + A^Au Y^t't' H ~T7~(Ad 2 A Ul — Ad 2 A u Y^t't'^Vb 

m b Vfb 



yt m b 'Xt s Vt'v A A ^ ^ , fyTV t * s 
t 



.,. t . A U2 Ad 2 ^t't^b'b + \ / T7T"^di y l«2^'tt'- (36) 

yt' m 6 A£ s Vtb v y*' 

where for later convenience we have dehned 

A Ul = Ad 1 = tan/3 , A U2 — Ad 2 — tan/3 + cot/3 . (37) 

In all the cases where the new physics contributions do not involve new operators (and in which 
Cfc ew (^o) = for k = 1,2,3,5,6 as in our case - see Eq. [32]), it is straightforward to incorporate the extra 
terms to the NLO formulae. In particular, these contributions effectively modify K t given in Eq. [M] which in our 
4G2HDMs should be replaced by 

Kt K? + V -^K™ + K? + %^K? , (38) 
Vtb V t s v tb v ts 

where , K$ , K^ 1 and represent the W and charged-Higgs contributions to the b — > 57 amplitudes from t 
and t' loops (see Fig.Q]). In particular, K^f can be obtained simply by replacing (neglecting in(^-)) 

E Q (x t ) -> E (x t '), 

A Q (x t ) -> A Q (x t '), 

A^xt) -> Ai{xr), 

F Q {x t ) -> F (x t >), 

F^xt) -> F 1 (x t >), (39) 
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in eq. [25] On the other hand, and Kfi , which represent the net contributions to the b — > sj amplitude from 
charged-Higgs exchanges (with t and t' as the internal quark, respectively), can be obtained from Eq.[3H]by calculating 
the functions E (yi), A (yi), Axfa), F (yi) and Fi(y f ) (i = t or i = i'). The LO functions A (y t ) and A Q (y t >) are 
given by 



Mvt) = -2 6C^ eff (fio), F (y t ) = -2 5C ( ° )eff (no), 



Mw) = -2 «cf' o// (w>), F (yt>) 



-2 SC. 



(0)'e// 



Oo)- 



and the NLO functions Ai(y t ) and Ai(y t >) by 



Ai{ Vi ) = -2<5cj 1) (/i ), F 1 {y i ) = -2 5C^\n ). 
The NLO contributions to the Wilson coefficients (in our 4G2HDMs) are given by' 3 ' 



<5Cf )e// (Mo) = i= 1,2,3,5,6 



and 



E (y i )=5Ci 1) ( f , )=A Ui 



3yf - 2yi . ~7y? + 29y. 2 
-jlny 4 + 



6(1 - y*) 4 



36(1 - y*) 3 



(40) 
(41) 

(42) 
(43) 



= A r 



f 16y 4 - 74y? + my 
I 9(1 -ytf 



-1202y 4 + 7569y 3 



L« 2 

5436^ + 797y., 



1 

1 

Hi 



-liyf 



486(1 - Vl f 
135y 3 - 18y 2 - 31» 4 



27(1 - y,) 4 
14y 3 - 128y 2 + 66y, 

9(1 -y,) 4 
16y 3 - 94y 2 + 42y z ~ 

9(1 -y,) 3 



-63y 4 + 807y 3 - 46%? + 7y t 

81(1 -y,) 5 
6y 4 + 46yf - 28y 2 



i„4} 


+ A A { 


mf J 





9(1 -y*) 5 
-32y 3 



112y 2 - 48y ; 



- 52y 2 + 28y, 

3(1 -y.) 3 



9(1 -y,) 3 

-12y 3 -56y 2 + 32y, 



9(1 - yi) 4 



Li 2 
lny, 



1-1 

y» 



In 4 

mi 



(44) 



f 13y 4 -17y 3 + 30y 2 £ . / 1 
I 6(1 -y 4 ) 4 2 I y, 

4451y 4 + 7650y 3 - 18153y 2 + 1130y, 

1296(1 -y,) 4 
-7y 4 + 18y 3 - 261y 2 - 38y, 



36(1 - y,) 4 
34y 3 - 7y 2 + 165y ; 



-468y 4 + 321y 3 - 2155y 2 - 2 Vi 

216(1 -y,) 5 
-17y 3 - 31y 2 



In yi 



In yi 



1-4} 


+ M { 


. ™ 2 J 





6(1 -y,) 5 

17y 3 + 25y 2 - 36y, T . 



12(1 -y,) 4 
7y 3 - 16y 2 + 81y- 



hi yi 



29y 3 - 44y 2 + 143y, 

8(1 -y,) 3 



6(1 -y,) 3 
17yf + 19y, 

3(1 - y,) 4 



Li2 1 - 



Inyi 



111 



Mo 



(45) 



6(1 ™y,) 3 

The electroweak and non-perturbative corrections are retained to their SM predictions as given in (6j| (see also 
eq. [22]), i.e., we do not take into account the effect of our 4G2HDM on these corrections. 



[3] The NLO results for the Wilson coefficients in a 2HDM can be found in [5ll,[7ll. 
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2. B q — B q mixing 

In the SM, B q — B q mixing (q — d, s) proceeds to an excellent approximation only through the box diagrams with 
internal top quark exchanges. On the other hand, in our 4G2HDMs there are additional contributions to B q — B q 
mixing coming from the loop exchanges of the t' and charged-Higgs. 

In the 4G2HDM, the mass difference AM, = 2\M 12q \ is given at LO by [4] 

G 2 

M 12q = ^M^f Bq B q M Bq [AW + M HH + M HW ] , (46) 

where we have used 

(B q \(sb) (v _ A) (sb) (v _ A) \B q ) = y%B q Ml^ (47) 
(B q \(sb) (s+P) (sb) (s+P) \B q ) = - b -f B B q M Bq . (48) 

and 



M hw 



X\ q VttSww(xt) + Vt't'Swwixt') + 2 \ t b q X t bq r]tt'Sww{xt 1 xt'), 



t > 2 



K q * s HH{yt) + K q s H H(yt') + 2 xl q Xb q S H H{yt,yt'), 

Kq SHw{xt,z) + \ bq Suw{xt',z) + 2 \ bq \ bg SHw(%t,xt>,z), 



(49) 



t^, Xi — — Vi = (i = t or t') and A ^ r , = V*, V u d ■ 

The contributions from VT-exchange diagrams with qi and qj (i,j are generation indices) as the internal quarks are 
given by, 



Sww{Xi,Xj^ — XiXj 



1 3 1 



4 2(1-1,-) 4(1-.t,) 2 



lnx,- , . 3 



4(l-.T l )(l-X J ) 



and Sww{xi) = SVwK^ij ^i) can be obtained from Eq. [50]by taking the limit Xj — > X{ 
The contributions from the iJ + -exchange diagrams are given by 



(50) 



SHH{yt,yt') = zS Li Sl 2 



S " Hi f yt ' ] B Ll B L2 lx b S^ H {y u y t ,)B Rx B R2 



SHH{yt) = zSl 2 
S H H(yt>) = zS 2 Li 



Bl 2 ~ eX b S% H ( yt )B R2 



- A B L! ~ z Xbb HH\yt')B Rl 



where Xb 



,2 Mi 



m H+ m b(m b ) 2 ' 



(51) 



SHH(yi,yj) = ViVj 



1 / yflnyt Vj^yj 



(Vi - vs) \ (i - y,) 2 (i - y 3 ) 2 / (i - w)(i - %) ' 



Sfiniyi) — ^HH{yiTyj)vj^yi 



1-2 



s HH(y^yj) = vm 



y t In yi yjlnyj 



(Vi-yj) \(l~y t ) 2 (1-y,) 2 

^Hniyi) — ^HH(yi:yj)tj.,^yi- 



(1-2/0(1-%) 



(52) 

(53) 
(54) 
(55) 



[4] The LO results for B q — B q mixing in a "standard" 2HDM of type II with three generations of fermion doublets are given in [72| , 
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and the terms 





— A Ul 


+ A U2 Ti t 't 


, m 4 V t b , , 

+ A« 2 TT - ^tt 

m*' VW 


Bl 2 


A 

— M-Ux 


+ A U2 Htt 


+ Az 2 rr—l^t't, 

m t V tb 


O 


A 

— /MJj 




A m t Vt*s v 

T /1« 2 777" -^t 


Sl 2 


= ~A U1 


+ A U2 H tt 


+ A U2 TFJT^t't, 

mt F t * 


Br 1 


= A dl - 


Ad 2 ^bb — 




Br 2 


= A dl - 




. my Vtfe/ 

TO b Vtb 



(56) 



are obtained from the b — > t, t and t,v — > s vertices in the box diagrams. 

The functions Snw( x i, x j, z ) obtained from diagrams with both W and _ff + -exchanges are given by 



S H w(xt,x t ',z) = 2x t x t '(S Ll B L2 + S L2 B Ll ) 

Si(x t ,z) 



Sx{x t ,Xf,z) 



+ S 2 (x t ,x t >,z) 



Shw{x u z) = 2x 2 t S L . 2 B L . 2 
S HW {x t ',z) = 2x 2 t ,S Ll B Ll 



4 

Si(x t ',z) 



+ S 2 (x t ,z) 
+ S 2 {x v ,z) 



(57) 
(58) 
(59) 



where 



Si (xi , Xj , z) 
S 2 (xi,Xj,z) 



; In ; 



+ 



00 j I11 00 j 



(1 - z)(z - Xi)(z - Xj) (1 - Xi){Xi - z)(Xi - Xj) (1 - Xj)(Xj - z)(Xj - Xi)' 

z 2 lnz x 2 \riXi xHnx-, 



(1 - Z)(z — Xi)(z — Xj) (1 - Xi)(Xi — z)(Xi — Xj) (1 - Xj)(xj — z)(Xj 



(60) 



and the functions S\(xi, z) and S 2 (xi, z) can be derived from the expressions for Si(Xi, Xj, z) and S 2 (xi, Xj, z), respec- 
tively, by taking the limit Xj — > Xi. 

3. Combined constraints 

Using the analysis above, we derive below the constraints on our 4G2HDMs that come from Br(B — > X s -f) and 
AM q (q = d,s). For the B-physics parameters we use the inputs given in Table HI As an illustration, the 4th 
generation quark masses are fixed to m t < = 500 GeV and rrib> = 450 GeV, consistent with the direct limits from the 
Tevatron [32j and the perturbative unitarity upper bounds [HI, [73| . ^ We vary the charged Higgs mass in the range 
200 GeV < m H + < 1 TeV and study the dependence on e t in the range < e t < 1, while fixing et — nif, / 'm^ (~ 0.01). 
We also vary the 4x4 CKM element Vt'b in the range < |Vt/&| < 0.2 (see also next section), keeping |A' h | < 0.02 
and varying tan/3 in the range, 1 < tan/3 < 30. We made a scan over the entire parameter space by a flat random 
number generator and obtained bounds and correlations among the various parameters mentioned above. 

Let us first consider the case Vt'b — ► 0, corresponding to the "3+1" scenario, in which the 4th generation quarks 
do not mix with the quarks of the 1st three generations (we assume that \Vfb\ >> \Vt' s \,\Vt'd\)- In this case, the 
top-quark loops become dominant, since contributions to the amplitudes of B — ¥ X s j and B q -B q mixing from f'-loops 
are mostly suppressed apart from the terms which are proportional to (% /mt) • \\ s (see Eqs. 131)1 and [5S|) . 

In Figs. [U [3] and H] we plot the allowed ranges in the m#+ — tan /3 (left plots) and the tan/3 — (right plots) planes, 
in the 4G2HDM of types I, II and III, respectively, using \Vfb\ — 0.001 (with |A^ h | = 10~ 5 correspondingly). 



[5] There is a very weak dependence of B — > X s "f and B — B-mixing on the ft'-mass, since it enters only in the the H + ud Yukawa couplings 
with no dynamical and/or kinematical dependence. 
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fBT d = 0.224 ± 0.015 GeV [74, 75] 
f = 1.232 ±0.042 [74, 75] 
?? t = 0.5765 ± 0.0065 [76J 
AM S = (17.77 ±0.12)ps _1 
AMd = (0.507 ± 0.005)ps _1 
f B = (0.208 ± 0.008) GeV 



pole 



(170 ± 4) GeV 



\V ub \ = (32.8 ±2.6) x 10" 4 a 
\V cb \ = (40.86 ±1.0) x 10~ s 
7 = (73.0 ± 13.0)° 
BTZ(B -> X 3 j) = (3.55 ± 0.25) x 10" 
m b (m b ) = 4.23 GeV 
a s (Mz)=O.U 
t b + = 1.63ps 
m T = 1.77 GeV 



"It is the weighted average of V^™ ! = (40.1 ± 2.7 ± 4.0) X 10 4 and V^ 1 = (29.7 ± 3.1) X 10 4 . In our numerical analysis, we increase 
the error on V ub by 50% and take the total error to be around 12% due to the appreciable disagreement between the two determinations. 



TABLE I: Inputs used in order to constrain the 4G2HDM parameter space, 
from Particle Data Group 49]. 



When not explicitly stated, we take the inputs 




FIG. 2: The allowed parameter space in the m H + — tan /3 and tan ft — e t planes, following constraints from B — > X s j and B q -B q 
mixing, in the 4G2HDM-I, for V t 'b = 0.001, m t i — 500 GeV, m h i = 450 GeV and et = rut/my . 



We see that in the type-I 4G2HDM, the "3+1" scenario typically imposes tan/3 ~ 1 with et typically larger than 
about 0.4 when ;$ 500 GeV. In particular, for a fixed et the upper bound on tan/? is reduced with the charged 
Higgs mass, allowing m H + <L 200 GeV for tan/3 ~ 1 and restricting m H + ^ 500 GeV for tan/3 ^ 1.5. In the type II 
and type III 4G2HDMs we observe a similar correlation between tan/3 and rn^+, however, larger tan/3 are allowed 
for e t m t / m t ' and a charged Higgs mass typically heavier than 400 GeV. 

Let us now turn to the case of a Cabbibo size mixing between the 4th and 3rd generation quarks, setting \Vfb\ — 
\Vtb>\ = 0.2. In Fig. [S] we show the allowed parameter space in the tan/3 — et plane in the 4G2HDM-I, II and III 
with \V t 'b\ = 0.2, ra v = 500 GeV, ra b < = 450 GeV and e b = m b /m b >. In addition, we take |A*' b | = 0.004 for Type-I 
and 0.001 for Type-II and III models and depict these correlations for two different values of the charged Higgs mass: 
M H + = 400 and 750 GeV. In the type II and type III 4G2HDMs we see a similar behavior as in the no mixing case 
(Vfb 0), while in the 4G2HDM-I we see that "turning on" Vt'b allows for a slightly larger tan /3, i.e., up to tan/3 ~ 5 




FIG. 3: Same as Fig. H for the 4G2HDM-II. 
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FIG. 4: Same as Fig. \% for the 4G2HDM-III. 



for e t £ 0.9. 

With a similar set of inputs, setting now et ~ rati W > m Figs. [5] and [7] we plot tan/3 as a function of M#+ (where 
| A' J is kept free) and of A£ s (where M H + is kept free), respectively, in the three different types of our 4G2HDMs. 
We note that, similar to the no mixing case, larger values of tan/3 are allowed in the 4G2HDM of types II and III. 
Furthermore, m H + ~ 300 GeV and tan/3 ~ 1 are allowed in the 4G2HDM-I, and from Fig. [5] we see that |A£J up to 
0.01 is allowed in the case of the 4G2HDM-I and II, while in 4G2HDM-III |A*' 6 | < 0.005 is typically required. 




FIG. 5: Allowed parameter space in the tan/3 — et plane in the 4G2HDM of type-I (left), type-II (middle) and type-Ill (right), 
for \V t > h \ = 0.2, nv = 500 GeV, m v = 450 GeV, e b = m b /m v , |A*' b | = 0.004 and with m H + = 400 and 750 GeV. 




FIG. 6: Allowed parameter space in the m H + —tan/? plane in the 4G2HDM of type-I (left), type-II (middle) and type-Ill (right), 
for \V t > b \ = 0.2, m t > = 500 GeV, m b , = 450 GeV, e b = m b /m b ,, \\^ b \ = 0.004 and e t = 0.34(~ rn t /m t ,). 
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V t ' • ■ « . :.. . 




£, = 0.34 



v'i -v. 1 ,** * 




0.004 OX 



0.008 0.01 




FIG. 7: Allowed parameter space in the tan /3 — | A* b | plane in the 4G2HDM of type-I (left), type-II (middle) and type-Ill (right), 
for a fixed \V t / b \ = 0.2, e± ~ m t /m t i and for m t i = 500 Ge V, m b / = 450 Gel/ and — mi/my . 



To summarize this section, we find that the parameter space of our 4G2HDMs, when subject to constraints from 
Br(B — > X s j) and B q — B q mixing, can be characterized by the following features: 

• In the type II and III 4G2HDMs large tan j3 > 20 are allowed for e t £ 0.1. 

• In the 4G2HDM-I tan/3 is typically restricted to be tan/3 ~ O(l) with e t ~ m t /m t i, reaching at most tan/3 ~ 5 
if e t ~ 1 and Vfb ~ O(0. 1), i.e., of the size of the Cabbibo angle. 

• The charged Higgs mass is typically heavier than about 400 GeV in the type II and III 4G2HDM and is allowed 
to be as light as 200-300 GeV (depending on Vfb) in the 4G2HDM-I. In all three models the lower bound on 
m H + increases (typically linearly) with tan j3; reaching m H + ~ 1 TeV already for tan j3 ~ 2 in the 4G2HDM-I 
and tan (3 ~ 7 in the type II and III 4G2HDMs if e t — m t /m t > . 



• In the 4G2HDM-III, |A*' b | < 0.005 is required if e t - m t /m t >, but values up to |A*' 6 | 
the 4G2HDMs of types I and II. 



0.01 are still allowed in 



Constraints from Z 



bb 



It has been long known that the decay Z — > bb is very sensitive to effects of new heavy particles, in particular, to 
the dynamics of multi- Higgs models through loop exchanges of both neutral and charged Higgs particles (see e.g., 
[73, uM)- The Zbb vertex can be parameterized as follows: 



V qqZ = -i—q'Yu (g q LL + g q nR) qZ^ , 

Cw 

where sw{cw) = sm9w(cos9w): L(R) = (1 — (+)7s) /2 and 

-= n SM 1 new 

g q L,R - g qL ,R + g q L,R > 

so that Qqjf,R are the SM (1-loop) quantities and g q [ w R are the new physics 1-loop corrections. 
The effects of the new physics, ff^f^j, is best studied via the well measured quantity i?^: 



(61) 



(62) 



Rh — 



T(Z -> bb) 
T(Z -> hadrons) ' 



(63) 



which is a rather clean test of the SM. In particular, being a ratio between two hadronic rates, most of the electroweak, 
oblique and QCD corrections cancel between numerator and denumerator. 

Following the analysis in !77| , we parameterize the effects of new physics in Ri, in terms of the corrections 5b and 
S c to the decays Z — » bb and Z — > cc, respectively: 



(64) 
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where i?f M = 0.21578 ±0.00005 and Rf M = 0.17224 ±0.00003 [z^ are the corresponding 1-loop quantities calculated 
in the SM, and S q are the new physics corrections defined in terms of the Zqq couplings as: 

n SM n new _r_ n SM n new 
<■ _ Q yqL yqL "T ilqR ilqR 





FIG. 8: One-loop diagrams for corrections to Z — > d/dj /rom charged Higgs loops. 



With the new scalar-fermion interactions in Eqs. [DO the corrections to Rb from a 4th generation quarks in our 
4G2HDMs are of three types: (i) the SM4-like corrections due to the 1-loop W — t' exchanges (see also flU l80l l8ll|). 
(ii) the 1-loop diagrams in Fig. [5] involving the H + — if exchanges and (iii) the 1-loop corrections involving the FC 
W a bb' interactions (coming from the non-diagonal 34 and 43 elements in E d ), where H° = h, H or A. 

Let us first consider the SM4-like (non-decoupling) correction to i?f,, i.e., from the 1-loop diagrams involving 

the W — if exchanges (which are also present in our 4G2HDMs) . It is given by [lj], : 

SM4 _ 9 2 ( m l m 1 \ -2 n (f .^ 

9 " L -ei^^-^lJ 8111 034 ' (66) 

where ^34 is the mixing angle between the 3rd and 4th generation quarks, i.e., defining \Vfb\ — \Vtb' \ = sin034. This 
SM4-like effect on Rt is plotted in Fig. |H1 We see that Rb puts rather stringent constraints on the me — #34 plane which 
is the SM4 subspace of the parameter space of our 4G2HDMs. In particular, increasing the if mass would tighten 
the constraints on 6*34; e.g., for me ~ 500 GeV the t' — b mixing angle is restricted to #34 ;$ 0.2. The upper bound 
on #34 stays roughly the same in our 4G2HDMs where the effects from the charged Higgs loops are included. For 
concreteness, for the rest of this section we will fix #34 to either #34 = or 6*34 = 0.1, 0.2, representing the no-mixing 
or mixing cases. 

Using the generic formula given in [82| , we calculated the 1-loop corrections to Rb from the charged-Higgs and from 
the FC neutral-Higgs exchanges and found that: 

• In all three models, i.e., 4G2HDM-I,II,III, S c <C Sb, so that we can safely neglect the new effects in Z —> cc. 

• The 1-loop FC neutral-Higgs contributions are much smaller than the 1-loop charged-Higgs contributions shown 
in Fig. [SI in particular for ej, < 1. We, therefore, focus below only on the leading effects coming from the 
charged-Higgs sector. 
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FIG. 9: Upper plots: Rt in the SM4, as a function 0/834 for several values of the t' mass (left) and as a function of m t i for 
034, = 0.1 and 0.2 (right). Lower plots: Rb in the 4G2HDM-I, as a function of the charged Higgs mass for m t i — 500 GeV, 
e t =m t /m t i and for (tan /3, #34) = (1, 0), (1, 0.2), (5, 0), (5, 0.2) (left), and as a function of m t i for 834 = 0.2, e t = m t /m t > and 
for(m H + [GeV], tan /3) = (400, 1), (400, 5), (750, 1), (750, 5) (right). 



• The charged-Higgs interactions in models 4G2HDM-II and 4G2HDM-III have negligible effects on i?h and are, 
therefore, not constrained by this quantity. On the other hand, Rb is rather sensitive to the charged Higgs loop 
exchanges within our type I 4G2HDM. 

In light of the above findings, we plot in Fig. [9] the quantity Rb in the 4G2HDM-I (calculated from Eq. [64]) , as a 
function of the charged Higgs and t' masses, fixing et = m t /m t / and focusing on the values tan/3 = 1,5, 634 = 0,0.2 
and rriH+ = 400, 750 GeV. We see that, while there are no constraints from Rb on the charged Higgs and t' masses 
if tan/3 = 1, for higher values of tan/3 a more restricted region of the charged Higgs mass is allowed which again 
depends on # 34 , e.g., for tan /? = 5, 550 GeV ^ m H + ^ 800 GeV, and m t ' ^ 500 GeV is required in order for R b to be 
within its 2a measured value (R e b xp = 0.21629 ± 0.00066 (zl). 

In Fig. [TU] we show the allowed ranges in the m H + — tan /3 plane in the 4G2HDM-I, subject to the Rb constraint 
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FIG. 10: Allowed area in the m H + — tan/? in the 4G2HDM-I, subject to the Rt measurement (within 2a), for m t i = 500 
GeV, my = 450 GeV, #34 = 0.2, ej, = mi,/m b i and for three values of the t — t' mixing parameter: et = ej ~ 0.01 (left plot), 
et ~ m t /m t i ~ 0.35 (middle plot) and et = 1 (right plot). 



(2a), for tan /3 in the range 1-15, fixing m t / = 500 GeV, m b > = 450 GeV, #34 = 0.2, e b = m b /m b i (which also enters the 
t'bH + vertex) and for three representative values of the t — t' mixing parameter: et = e b ~ 0.01, e t — m t /m t i ~ 0.35 
and et — 1. As expected, when tan/3 is lowered, the constraints on the charged Higgs mass are weakened. We see 
e.g., that for e t — m t /m t ' ~ 0.35, tan/3 ~ 1 is compatible with m H + values ranging from 200 GeV up to the TeV 
scale, while for tan j3 ~ 5 the charged Higgs mass is restricted to be within the range 450 GeV ^ rn H + ^ 750 GeV. 
Note however, that in the 4G2HDM-I, tan/3 = 5 with e t = mt/m t > is not allowed by constraints from B-physics flavor 
data (see previous section). 



C. Constraints from the Oblique parameters 



The sensitivity of 4th generation fermions to PEWD within the minimal SM4 framework was extensively analyzed 
in the past decade [i^, [8(1 |83T - [87| . One of the immediate interesting consequences of the presence of the 4th gen- 
eration fermion doublet (with respect to the PEWD constraints) is that it allows for a considerably heavier Higgs, 
thus removing the slight tension between the LEPII bound on the mass of the SM Higgs mn ^ 115 GeV and the 
corresponding theoretical best fitted value (to PEWD) m# = 87^6 GeV |49| . In fact, a Higgs with ttih ~ 300 GeV 
becomes favored in the SM4 when m t < — my ~ 50 GeV and #34 is of the size of the Cabbibo angle, see e.g., [80l [85|. 
On the other hand, if, as in our case, the 4th generation fermions are embedded in a 2HDM framework, then there is 
a wider range of parameter space for which a lighter Higgs with a mass of 0(100) GeV is allowed (see [36[ and our 
analysis below). In addition, in the 2HDM case, the LEPII lower bound uih ~ 115 GeV can be relaxed, depending 
on the value of sin(a — j3) [sin(a — 0) = 1 corresponds to the current SM bound] which controls the ZZH coupling 
responsible for the Higgs production mechanism at LEP. 

In general, the contributions to the oblique parameters (S,T,U) of 4th generation fermions (ASf,ATf,AUf) and 
of extra scalars (A5 S , AT S , AU S ) are calculated with respect to the SM values and are bounded by a fit to PEWD 
& 

AS = S-Ssm =0.02 ±0.11 
AT = T — Tsm = 0.05 ± 0.12 

AU = U - U SM = 0.07 ± 0.12 , (67) 

where, following the fit made in [§s|, the SM values are defined for a Higgs mass reference value of M^ e ^ = 120 GeV 
and for m t = 173.2 GeV. The effects of our models (and in general of any heavy new physics) on the parameter U can 
be neglected. We, therefore, consider below the constraints from the 2-dimensional ellipse in the S — T plane which, 
for a given confidence level (CL), is defined by (see e.g., |23||): 

where S exp — 0.02 and T exp = 0.05 are the best fitted (central) values in Eq. [B71 as — 0.11, or = 0.12 are the 
corresponding standard deviations and p = 0.879 [88| is the (strong) correlation factor between S and T. 
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Note that the contribution of the Higgs spectrum of our 4G2HDMs to S and T are identical to that of any general 
2HDM. We thus use the analytical expressions given in (83|, where we also include in ATf the new contributions from 
the Wt'b and Wtb' off-diagonal CKM mixing angles (see e.g., |8(ij|): 

AT f = a 2 2 (\Vvv\ 2 *t>v + \V t > b \ 2 F t , b + \V w \ 2 F tb , ~ \V tb \ 2 F tb + ]-F liV \ , (69) 



8tts' w ^ w 



with 



Xi -f- Xj XjXj X{ 
F *3 = o - Z, — l0 &— > ( 70 ) 



X i X i 



and x k = (m k /m z ) 2 . 

We first "blindly" (randomly) scan our parameter space, varying them in the ranges: tan/3 < 30, #34 < 0.3, 
100 GeV < m h < 1 TeV, m h < m H < 1.5 TeV, 100 GeV < m A < 1 TeV, 400 GeV < m t ,,m b , < 600 GeV, 
100 GeV < m u <,m T i < 1.2 TeV,I 6 l and the CP-even neutral Higgs mixing angle in the range < a < 2tt. 

We use a sample of 100000 models (i.e., points in parameter space varied in the above specified ranges) and plot 
the result in Fig. [11] We find that out of the 100000 models about 3000 are within the 99%CL contour, 1500 within 
the 95%CL contour and 100 within the 68%CL contour. We compare these results to the SM4 case also shown in 
Fig. QT] (again using a sample of 100000 models), where the 4th generation quark and lepton masses as well as the 
(single) neutral Higgs mass are varied in the same ranges as specified above. We find that in the SM4 case only 
a few points (out of the 100000) are within the 68%CL S-T contour, while the number of SM4 points within the 
95%CL and 99%CL allowed contours are comparable to the 2HDM case. This quantifies the slight preferability of 
the 2HDM (with respect to the amount of fine tuning required for compatibility with the available precision data) as 
an underlying framework for a 4th generation model. 

We also examined the correlation in the m#+ — tan/3 plane, when subject to the PEWD S-T constraint. This is 
shown in Fig. Q21 where the data points are taken from the same 100000 sample used in Fig. [TT] (i.e., the rest of the 
parameter space was varied in the ranges specified above). We see that compatibility with PEWD mostly requires 
tan j3 ~ 0(1) with a small number of points in parameter space having tan/3 £ 5. 




FIG. 11: The allowed points in parameter space projected onto the 68%, 95% and 99% allowed contours in the S-T plane, in 
the 4G2HDMs (left) and in the SM4 (right). The data points are varied in the ranges: tan/3 < 30, 634, < 0.3, 100 GeV < mj, < 
1 TeV, m h <m H < 1.5 TeV, 100 GeV < m A < 1 TeV, 400 GeV < m t ,,m b , < 600 GeV, 100 GeV < m J/ -,m T , < 1.2 TeV and 
the CP-even neutral Higgs mixing angle in the range a £ 2ir. 



[6] Note that the perturbative unitarity upper bounds on the lepton masses are about twice larger than those on the quark masses [73j : 
thus allowing 4th generation lepton masses around 1 TeV. 
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FIG. 12: 95% CL allowed range in the m H + — tan /3 plane. The data points are varied as specified in Fig. 



Next we consider the correlations between the mass splitting among the 4th generation quark masses, Am,< = 
m t ' — my, and the lepton masses Amy = m„ 4 — mg i . In Fig. [lj|]we plot the 95%CL allowed regions (i.e., subject 
to the measured 95%CL contour in the S-T plane) for both the 4G2HDMs and the SM4 in the Am q > — Am^ plane, 
again using the same data set of 100000 models used in Fig. [IT] We see that, while in the SM4 case the allowed mass 
splittings are restricted to -100 GeV < Am q , < 100 GeV and -200 GeV < Amy < 200 GeV, in the 4G2HDMs these 
mass splitting ranges are significantly extended to: —200 GeV < Am,/ < 200 GeV and —500 GeV < Amp < 400 GeV. 




In Figs. [JJ] we again plot the 95%CL allowed regions in the Am q i — Ami: plane, for both the 4G2HDMs and the 
SM4, considering now the "3+1" scenario, i.e., with a vanishing mixing between the 4th generation quarks and the 
lighter three generations; 6*34 = 0. The rest of the parameter space is varied as in Fig. [13j We see that in the SM4 
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FIG. 14: Same as Fig. \13\ but for 834 = 0; the rest of the parameter space is varied as in Fig. \13i 



with #34 — ¥ there are no solutions where both the quark and lepton 4th generation doublets are degenerate, in 
particular, no solutions where both |Am g /| 50 GeV and |Am^| £ 100 GeV. On the other hand, the implications of 
the no-mixing case on the 4G2HDMs are mild as there are still points/models for which the 4th generation quarks 
and leptons are both almost degenerate. For such small (or no) 4th generation fermion mass splitting the amount of 
isospin breaking required to compensate for the effect of the extra fermions and Higgs particles on S and T is provided 
by a mass splitting among the Higgs particles, as is shown below. 

In order to demonstrate the interplay between the mass splittings in the Higgs and fermion sectors, we choose a more 
specific framework - partly motivated by our theoretical prejudice towards the possibility of dynamical EWSB, driven 
by the condensation(s) of the 4th generation fermions. In particular, we set a ~ 7r/2, for which case H ~ Re($") 
and h ~ Re($e); the heavier Higgs may be thus identified as a possible Q'Q' (Q' — t' , b') condensate, with a typical 
mass of mji ~ 2toq' . We thus set mjj = 1 TeV and take a nearly degenerate 4th generation quark doublet with 
m t > — 500 GeV and my — 490 GeV. We further study two representative values for tan/3: tan/3 = 1 and tan/3 = 5, 
recalling that for tan/3 ~ O(l), H + and A are roughly equal admixtures of and while if tan 2 /3 >> 1, one has 
H + ~ <j>^!~ and A ~ Im($°). The charged Higgs mass is set to m#+ = 600 GeV, so that it is within the Rb constraints 
for both tan (3 = 1 and tan j3 = 5 when ;$ e t ,$ (see previous section) . For simplicity we furthermore set #34 = 

and vary the 4th generation lepton masses in the range 100 GeV ,$ m v i,m T i ^ 1.2 TeV and the masses of the neutral 
Higgs particles, h and A, in the range 100 GeV ,$ mh, rriA ~ 1000 GeV. 

Using the above set of assumptions on our parameter space, we plot in Figs. [TBI and [TBI the 95%CL allowed region 
in the Am/' — mh, the m/, — tua and the Am^ — (m^ — mX) planes, using again a sample of 100000 models with 
tan/3 = 1 and tan/3 = 5, respectively. Under the above set of inputs, we find the following noticeable features: 

• There are allowed sets of points in parameter space (i.e., models) where both the 4th generation quarks and 
leptons are nearly degenerate with a mass splitting smaller than 50 GeV. These solutions require rrih and tjia 
to have a mass splitting smaller than about 400 GeV and to be within the narrow black bands in the m/j — 
plane, as seen in Figs. IT51 and [TBI 

• There are allowed sets of points with a large splitting between the 4th generation leptons, \m v i — m T i\ > 300 
GeV. These cases require a large splitting also among rrih and wia', 400 GeV ^ |m^ — itia\ ~ 800 GeV if tan/3 = 1 
and 600 GeV < \m h - m A \ < 800 GeV if tan/3 = 5. 

• For tan /3 = 1, a splitting in the 4th generation lepton sector of \m u > — m T i \ > 200 GeV requires rrih to be larger 
than about 400 GeV. 




Finally, in Table UTI we give a list of interesting points (models) in parameter space (of our 4G2HDM of types I, II 
and III) that pass all the constraints considered in this chapter, i.e., from the S and T parameters, from Rb and from 
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FIG. 15: 95% CL allowed regions in the Am<< — plane (left), in the Am ( / — (nih — vtia) plane (middle) and in the — tua 
plane (right), for tan/3 = 1, m H+ = 600 GeV, #34 = 0, a ~ 7r/2, m t / = 500 Gel/ and m 6 / = 490 Gel/. 77ie lepton masses 
and Higgs masses are varied in the ranges: 100 GeV ^ m v i , m r i 1.2 TeV and 100 GeV ^ mh,mA ~ 1000 GeV. T/ie 
dote correspond to solutions with |Am^/| < 50 GeV, t/ie red dot to solutions with m T i — m„/ > 200 GeF and the blue dots to 
solutions with m„< — m T < > 200 GeV. 




B-physics flavor data. In particular, the list includes models with mass splittings between the up and down partners 
of both the 4th family quarks and leptons larger than 150 GeV, models with a light 100 — 200 GeV neutral Higgs, 
models with degenerate 4th generation doublets, models with a large inverted mass hierarchy in the quark doublet, 
i.e., vrity — m t ' > 150 GeV, models with a light charged Higgs with a mass smaller than 500 GeV, models with a 
Cabbibo size as well as an O(0. 01) size t' — b/t — b' mixing angle (i.e., #34). 



IV. PHENOMENOLOGY OF THE YUKAWA SECTOR IN THE 4G2HDM-I 



Although this paper is not aimed to explore in detail the phenomenological consequences of the modifications to 
the Higgs Yukawa interactions involving the 4th generation quarks in our 4G2HDMs, in order to give a feel for their 
importance for collider searches of the 4th generation fermions, we consider below some phenomenological aspects 
of the 4G2HDM-I which is defined by (ad, (3d, ce u , flu) = (0,1,0,1). Recall that in this case, the E mixing matrices 
simplify to (keeping terms up to 0(e 2 q ), q = b, t): 
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(71) 
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so that Y] u > d — if i or j 7^ 3,4. This leads to new interesting patterns (in flavor space) of the ri°qiqj Yukawa 
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a point requires |A^ b | ^10 5 . 

6 point requires et ~ mb/my in order to have BR(fe' — y tH + ) ~ O(l) (see Fig. 1191 

TABLE II: List of points (models) in parameter space for our 4G2HDMs of types I, II and III, allowed at 95%CL by PEWD 
and B-physics flavor data. The 2nd column denotes the model(s) for which the point is applicable. Points 1-3,14-16 and 24 have 
m t i — my > 150 GeV with a light CP-even Higgs of mass mi ^ 300 GeV, while points 4,5,27 have a large inverted splitting 
m;,/ — rn t i > 150 GeV with a heavier h. Points 6,7 and 17,18,28 have nearly degenerate 4th generation quark and lepton 
doublets, while points 22,23 have a nearly degenerate 4th generation quark doublet with a lepton doublet heavier than the 
quark doublet. Points 8,19 have my — m t i > mw and a light charged Higgs, while points 9,16 have m t i — my ~ 150 GeV with 
a light Higgs mass of m h ~ 100 GeV. Points 1,8,16,17,22,23,26,27,29,30 all have a large t' - b/t - b' mixing angle: 6» 34 £ 0.15. 
Finally, points 10,11 give BR(t' -> th) ~ 0(1) (see Fig. [171 in the next section), point 12 gives BR(t' -> bH + ) ~ 0(1) (see 
Fig. [TS]in the next section) and point 13 gives BR(6' tH+) ~ 0(1) (see Fig. [151 in the next section). 



interactions in Eqs. g][7J (H° = h, H, A). In particular, the most notable new features of the 4G2HDM-I are: 

1. There are no tree-level FC neutral currents (FCNC) among the quarks of the 1st, 2nd and 3rd generations. 
That is, no tree-level c —> u, s —> d transitions, as well as no t — > u, t — > c, b —> d and b — > s ones. 

2. There are no tree-level FCNC effects involving transitions between the quarks of the 4th generation and the 
1st and 2nd generations, i.e, no t' — > u, t' — > c, b' — > d and b' — > s transitions. This, makes the 4G2HDM-I 
compatible with all FCNC constraints coming from light meson mixings and decays, i.e., in the K and D systems. 

3. There are new potentially large tree-level FCNC effects in the H°qiqj couplings involving the 3rd and 4th 
generation quarks (i.e., i,j = 3,4), which can have drastic phenomenological consequences for high-energy 
collider searches of the 4th generation fermions, as we will further discuss below. In particular, the FC Wt't 
and H'b'b interactions are (taking a —> n/2): 
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FIG. 17: The branching ratio for the t' decay channels t' — > th, t' — > bW and t' — > b'W^ (W^ is either on-shell or off- 
shell depending on the b' mass), as a function of my for m t i = 500 GeV, et = mt/m t i, tan/? = 1 and (nth [GeV], #34) = 
(120,0.05), (120,0.2), (220,0.05), (220,0.2), as indicated. Also, a = tt/2 and m H+ > m t > , m A > m t > is assumed. 



C(ht't) - - 9 - — ejl + t% i'(R+^L]th, 
2 m w V V m t' J 

C(Ht't) = - S -^e t ^^ F( R+ ^L)tH 
2 mw tp \ mt' J 

C {AHt) = l g -^e t 1 -^t'(R-^L)tA, 
2m w tp \ mt' ) 

and similarly for the T-L°b'b interactions by changing t t —> £b (and an extra minus sign in the Ab'b coupling). 

We thus see that, if e t ~ m t /m t /, then the above couplings can become sizable, e.g., to the level that it might 
dominate the decay pattern of the t' (see below). In fact, we also expect large FC effects in b' — > b transitions 
since, even for a very small e& <~ mb /my, the FC hb'b and Ab'b Yukawa couplings can become sizable if e.g., 
tan/3 ~ 5, i.e., in which case they are cx ^p 4 -. 

4. The flavor diagonal interactions of the Higgs species with the up-quarks, W.°uu 7 are proportional to tan/3, 
thus being a factor of tan 2 j3 larger than the corresponding "conventional" 2HDMs couplings, for which these 
couplings are oc cot/3 (e.g., as in the 2HDM of type II which also underlies the supersymmetric Higgs sector). 
In particular, the V.°tt couplings in our 4G2HDM-I are given by: 



(72) 

(73) 
(74) 
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FIG. 18: The branching ratios for the decay t' —¥ bH + as a function of e t for m t i — 500 GeV, m H + = 400 GeV, tan/? = 1, 
e b = m b /m b ,, m h = 220 and 350 GeV and {my [GeV], 6*34) = (400,0.05), (400,0.2), (450,0.05), (450,0.2), as indicated. Also, 
a = 7r/2 and vtia > rn t i is assumed. 
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We see that the hit and Att Yukawa interactions are indeed enhanced by a factor of t\ relative to the conventional 
htt and Att couplings in multi-Higgs models (with no suppression from t — t' mixing parameter e t ). On the 
other hand, the ht't' and At't' couplings are suppressed by the t — if mixing parameter and by tp, respectively: 
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I raw tp 



(78) 

(79) 
(80) 



5. The charged Higgs couplings involving the 3rd and 4th generation quarks are completely altered by the presence 
of the £ matrix in Eq. [7l For instance, the H + t'b and H + ib' couplings have new terms proportional to Vtb and 
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m b ,=500 GeV , m h =220 GeV , m H+ =300 GeV , tan(3=l , e t =m/m t , 
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FIG. 19: The branching ratios for the decay channels b' — > bh, b' — > tH + , b' — > tW and b' — > t'W, as a func- 
tion of et for m b i — 500 GeV, = 220 GeV, m H + = 300 GeV, tan/3 = 1, e t — mt/m t i and (m t i [GeV], #34) = 
(400,0.05), (400,0.2), (450,0.05), (450,0.2), as indicated. Also, a = ty/2 and m A > my is assumed. 



Vfb'- In particular, in the "3+1" scenario where V t 'di,V Ui b> — > for i = 1, 2, 3, we have: 



C(H+t'b) 
C(H+tb') 



V2mw 
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\/2mw 



tpll + t- 2 ) F (m t e t V tb L - mve b V t >b>R) bH^ 



tp ( 1 + ty ) t{m! t e* t Vt>b>L - m b e* b V tb R) b'H 



.'TT+ 



(81) 
(82) 



Recall that in the standard 2HDM of type II that also underlies supersymmetry (assuming four generations of 
fermions) the t' R bLH + would be oc m v V t 'b/ti3. We thus see that in our 4G2HDM-I the t' R bLH + coupling is 
potentially enhanced by a factor of i| • e t • (m t /m t >) ■ (V t b/V t >b)- For example, if tp = 3, m v ~ 500 GeV and 
tt ~ mt/mt' we get a factor of V~t b /Vt' b enhancement to the i' R bLH + interaction. 

The implications of the above new Yukawa interactions can be far reaching with regard to the decay patterns of the 
t' and the b' and the search strategies for these heavy quarks. In particular, in Fig. [T7]we plot the branching ratios 
of the leading t' decay channels (assuming m^+jm^ > m t /): t' —> th, bW, b'W^ [W^ stands for either on-shell or 
off-shell W depending on the mv], as a function of the b' mass. We use mf — 500 GeV, tan/3 = 1, e t = m t /m t ' and 
the following values for m h and (9 34 : (m h [GeV],6» 34 ) = (120, 0.05), (120, 0.2), (220, 0.05), (220, 0.2). We see that the 
BR(t' — > th) can easily reach 0(1), in particular when m# — m b i < myy and even for a rather large 634 ~ 0.2; see 
e.g., points 10 and 11 in Table [U for which BR(t' -> th) - 0{l). 

In Fig.[TH]we take m H + — 400 GeV (again assuming rriA > mp so that tf — > tA is still kinematically closed) and plot 
BR(t' — > bH + ) as a function of e t , for m t > = 500 GeV, tan/3 = 1, e b = m b /m b > ~ 0.01 mh — 200 and 350 GeV and the 
following values for m v and 34 [m b , [GeV], 634) = (400, 0.05), (400, 0.2), (450, 0.05), (450, 0.2). We see that the decay 
channel t' — > bH + can become important and even dominate if e t ~ fn t /m t i , in particular, when mt> — my < rayy and 
a small mixing angle of 6» 34 - 0(0.05); see e.g., point 12 in Table |ll] for which BR(t' -> bH+) - O(l). 
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In Fig. [19] we plot the branching ratios of the leading 6' decay channels, as a function of for my — 500 GeV, 
tan/3 = 1, m H + = 300 GeV, m h — 220 GeV, e t — m t /m t , and the following values for m t , and 8 3i (ra v [GeV], #34) = 
(400,0.05), (400,0.2), (450,0.05), (450,0.2). We see that in the 6' case the dominance of b' -> tH~ (if kinematically 
allowed) is much more pronounced due to the expected smallness of the b — b' mixing parameter, ej,, which controls 
the FC decay b' -> bh; see e.g., point 13 in TableUfor which BR{V -> bH~) ~ 0(1). 

This change in the decay pattern of the 4th generation quarks can have important consequences for collider searches 
of these heavy fermions. For example, as was already noticed in pjij . if t' — > th dominates then t' production at the 
LHC via gg —> t'P will lead to the dramatic signature of tihh. If mt < 2mw (so that h decays to 66) this will 
give a 66 + 2W signature (i.e., after the top decays via t — > bW), while if m/, > 2mw, 2m z the tihh final state can 
lead to either tihh — > tiW + W~ and /or tihh — > tiZZ. In particular, notice that the former tiW + W~ is the one 
conventionally used for b' searches [32|, while the latter will lead to e.g., a 2b + 2W + 4£ signature which is expected 
to have a rather small irreducible SM-like background (e.g., coming from gg — » tih) that can be further controlled 
using the kinematic features of the process gg — > t'F — > tihh — > tiZZ. If, on the other hand, t' — > bH + dominates, 
then the signature bbH + H~ should be focused on. In this case the t' searches will depend on the H + decays, e.g., 
H + -> tb or H + -> tv, which will lead to gg -> t'i' -> 66 + 2W or gg -> t'F -> 26 + 2r + $ T , respectively. 

For the 6' the situation is similar, i.e, the new decays 6' — > bh and/or 6' — > tH~ can also alter the search strategies 
for 6'. For example, if 6' — > tH~~ dominates the 6' decays, then gg — > b'b' will lead to, e.g., a tiH~H + — > At + 26 
signature as opposed to the "standard" 2t + 2W one when the 6' decays via 6' — > tW [901 • 

Clearly, these new 4th generation quark signatures deserve a detailed investigation which is beyond the scope of 
this paper and will be considered elsewhere [91| . 




We have introduced a class of 2HDMs, which we named the 4G2HDM of types I, II and III. Our models are 
"designed" to give an effective low-energy description for the apparent heaviness of the 4th generation fermions and 
to address the possibility of dynamical EWSB which is driven by the condensates of these new heavy fermionic states. 
This is done by giving a special status to the 4th family fermions which are coupled to the scalar doublet that has 
the heavier VEV. Such setups give rise to very distinct Yukawa textures which can have drastic implications on the 
phenomenology of 4th generation fermions systems. We studied the constraints from PEWD and from flavor physics 
in B-systems and outlined the allowed parameter space of our 4G2HDMs, which we find to have various different 
features than the simpler SM4 version with a single Higgs boson and a 4th family of fermions. For example, we find 
that the mass splitting my — my and the inverted mass splitting my — my can be as large as 200 GeV, and that the 
mass splitting in the 4th generation lepton doublet can be as large as 400 GeV. 

We focused on the 4G2HDM-I, where the Higgs doublet with the heavier VEV is coupled only to the 4th generation 
doublet while the "lighter" Higgs doublet is coupled to all other quarks. This model is, in our view, somewhat better 
motivated as it provides a more natural setup in the leptonic sector, i.e., addressing the existence of a 4th generation 
EW-scale neutrino. In addition, it has very distinctive features in flavor space: there are no tree-level FCNC among 
the 1st three generation of fermions as well as no FCNC among the 4th generation fermions and the light fermions of 
the 1st and 2nd generations. On the other hand, the 4G2HDM-I does give rise to potentially large tree-level FCNC 
t' — > t and 6' — > 6 transitions, which, as we briefly explored in the paper, can have significant implications on the 
search for the 4th generation quarks at high-energy colliders. For example, the FC decay t' — > th can become the 
dominant t' decay channel and should therefore effect the search strategy for the t! . 

Finally, we note that the 4G2HDM setups can also alter the production and decay patterns of the Higgs particles at 
hadron colliders. For example, the di-photon Higgs channel gg — > h — > 77 can be dramatically enhanced or suppressed 
(to the level of being unobservable at the LHC) compared to the SM4 case. The phenomenology of the production 
and decay channels of the Higgs particles in the 4G2HDMs will be considered elsewhere. 
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